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ABSTRACT 

The Educational Resources Information Center (ERIC) 
is one of the major bibliographic databases in the world. The ERIC 
Clearinghouse on Tests, Measurement, and Evaluation (ERIC/TM) 
contributes to this database by acquiring, selecting, and processing 
documents pertaining to all aspects of testing, evaluation, and 
learning theory. The ERIC/TM has become a vital resource within its 
area of expertise. The scope of the ERIC/TM includes tests and other 
measurement devices, and documents that discuss such instruments; 
methodology of measurement and evaluation; evaluation of programs, 
projects, and procedures; research design and methodology; human 
development; and learning theory. Improvements needed by the ERIC 
system are: (1) serving a wider audience within the educational 
community; (2) expanding dissemination activities for this audience; 
(3) collaborating more closely with existing systems and networks; 
and (4) controlling quality. Recent improvements to the ERIC/TM have 
been in the areas of improved document acquisition and processing; 
better quality and more relevant products; and improved dissemination 
of information. The ERIC/TM is establishing a structure to make all 
these activities possible; the cooperation and involvement of the 
measurement community is essential. (SLD) 
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ERIC/TM -- A Grow* ig Resource 



Lawrence M. Rudner & Lauress L. Wise 
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Since its inception in 1966, the Educational Resources Information Center (ERIC) 
has become one of the major bibliographic databases in the world. ERIC has acquired, 
reviewed, and processed more than one-half million citations that policymakers, program 
planners, researchers, and other users can readily identify and obtain. 

With ERIC, people can identify a host of citations that potentially meet their 
needs by using 

dial-up information services, such as Dialog, BRS, and SDC; 

the CD-ROM systems of ERIC, such as Silverplatter, OCLC, 
and OnDisc; and 

printed catalogues of abstracts. 

ERIC users csn review abstracts, order microfiche, order complete documents, or 
examine microfiched documents in any of over 750 locations in the United States and 92 
foreign countries. 

As the major bibliographic database for education, ERIC has become a vital 
resource for research, policy, and practice. The ERIC Clearinghouse on Tests, 
Measurement, and Evaluation (ERIC/TM) contributes to this database by acquiring, 
selecting, and processing documents that pertain to all aspects of testing, evaluation, and 
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learning theory. As with the larger ERIC system, ERIC/TM has become a vital resource 
within its area of expertise. 

Because ERIC is a mature system, its growing pains are in the distant pa^t. ERIC 
provides valuable services by continuing to do well what it has always done. But this 
does not mean that there are no opportunities for improvement or for doing new things. 
Despite ERIC/TM's record as a vital resource, major innovations and improvements can 
and should be made. ERIQTM must be on the leading edge of developments in the 
areas of its interests. Maturity need not inhibit further development; it can be a 
foundation on which to build new and valuable products and services. 

No revolution is called for; rather, the need is for improvements in document 
acquisition and processing and for new and creative dissemination activities, as called for 
by Bencivenga (1987\ Parsons (1986), and others. This paper describes problems 
confronting the ERIC system and recent enhancements to ERIC/TM. We hope that this 
paper will prompt others to communicate their ide&s about ways in which ERIC can 
better serve the educational community. We begin with some background on the history 
of ERIC, components of ERIC, and the role of ERIC/TM. 



An Overview of ERIC 



A Brief History of ERIC 



When ERIC was started, the first objective was to collect and catalogue 
unpublished literature that was not generally available to researchers, policymakers, and 
practitioners. ERIC was based on two predecessors - the National Technical 
Information Service (NTIS) and the Defense Technical Information Center (DTIC). 
These organizations were established during and shortly after World War II. They were 
created to handle the flood of domestic and foreign technical reports that were beyond 
the scope of conventional libraries. 

The original impetus for ERIC was the government's need to track its projects. 
In the 1960's, the Department of Health, Education and Welfare funded numerous 
research studies, exemplary programs, and demonstration projects. These projects often 
ended with final reports that were filed in government cabinets, thus limiting access to 
them. Because the results of these studies were not widely disseminated, the government 
periodically funded projects that reinvented earlier studies (Hoover and Brandhorst, 
1982). 

Between April 1964 and March 1966, ERIC began storing documents in the same 
way that a traditional library stores documents. During this period, documents were 
microfiche d and the ERIC Document Reproduction Service began. It was also during 
this period that the ERIC system was designed (Trester, 1981). 

During 1966, the government granted contracts to create a central editorial and 
computer facility and 12 clearinghouses. Also during 1966, the first thesaurus was 
developed and the first abstract journal (Research in Education) was printed. During the 
following year, six more clearinghouses were added to the system and the word 
"Research" in ERIC was changed to "Resources." In 1968, ERIC clearinghouses began 
producing their first Information Analysis Products. 
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ihc systematic acquisition, abstracting, and cataloguing of unpublished literature 
pointed to the need to do the same with published educational literature. In 1969, the 
Current Index to Journals in Education (CUE) began to be published. 

The final major component of the ERIC system, private-sector database vendors, 
was added in 1972, Lockheed established the DIALOG on-line retrieval system with 
ERIC as its first file. Since that time, DIALOG has been quite successful. ERIC is one 
of its most frequently searched databases, and it retains its position as the first file in the 
system. ERIC is also now available on several other on-line systems as well as on 
compact disk (CD-ROM). 

By the end of 1972, all of the basic acquisition and processing components of the 
ERIC system were in place. Most of the significant changes in the ERIC system since 
then have been refinements in procedures, products, and focus. For more information, 
see A Cumulative List of Significant Changes in the ERIC System 1966-1985, compiled by 
the ERIC Processing and Reference Facility, February 1986. 



ERIC Today 



Today, ERIC performs the following functions: 

Acquiring documents - ERIC has established a 
nationwide network to obtain copies of virtually all 
significant documents related to education. 

Selecting and screening documents - ERIC 
follows detailed guidelines to achieve quality 
control. ERIC selects approximately one 
document from every three that it acquires. 

Cataloguing, indexing, and abstracting - 
ERIC's documents receive detailed document 
descriptors and abstracts* 

Developing a thesaurus (lexicography) - All ERIC 
components contribute to developing and refining the 
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ERIC Thesaurus, the authority list used in indexing 
and searching. 

Generating and maintaining a database - ERIC 
abstracts and descriptors are compiled into a database 
that is available at a nominal charge. 

Producing journals of abstracts - ERIC 
publishes two monthly journals of abstracts, 
Resources in Education and Current Index to 
Journals in Education. 

Developing information analysis products - All ERIC 
clearinghouses produce information analyses that 
synthesize information on a specific issue. 

Delivering documents - ERIC arranges for its docu- 
ments to be microfiched and reproduced in hard copy. 

Creating public awareness - ERIC staff participates in 
a wide range of professional meetings to make the 
system, its products, and its services more widely 
known. 

Helping users - All clearinghouses provide user 
services in the form of searches, workshops, and 
responses to inquiries. 

Providing technical assistance - ER T C staff 
regularly provides technical assistance to 
organizations, such as state and foreign 
governments, who are interested in aspects of 
the ERIC system. 



ERIC's Components and their Interrelationships 



The ERIC system is well known and used. Of the 300 databases that Dialog 
Information Services offers, ERIC has became one of the most frequently accessed. In 
1983, the References and Adult Services Division of the American Library Association 
voted ERIC as the most helpful on-line database. 

ERIC has achieved this stature because it uniquely joins the federal government 
with private enterprise and subject-matter experts. The main components of the ERIC 
system are: 

central FRIC, which establishes policy and administers the 
entire system, 

16 clearinghouses, which acquire and prepare documents, 

the ERIC Processing and Reference Facility, which prepares 
the ERIC database, 

the ERIC Document Reproduction Service (EDRS), which 
makes microfiche and hard copy available, and 

database vendors, who add information retrieval systems or 
print materials and make the database commercially 
available. 



The federal government spends approximately $5 million a year for the clearing- 
houses and the ERIC Facility. This federal investment, in turn, supports $136 million of 
activities (Heinmiller, 1S31). The government does not fund EDRS or the database 
vendors. Rather, it awards no-cost-to-the-government contracts for document repro- 
duction and publication of CUE, and it encourages on-line database vendors to include 
the ERIC database in their systems. These agencies provide services and market 
products to users to cover their costs and make a profit. 



CgPtral ERIC (CERIC) 



Central ERIC contains fewer than 15 people within the Information 
Services Program of the Office of Educational Research and 
Improvement (OERI). CERIC develops the statements of work, 
lets the contracts, selects contractors, funds the operations of the 
contractors, and monitors the contracted work for quality, timeliness 
and completeness. CERIC establishes all basic policies and 
directions for the system. 

Clearinghouses 

Materials are acquired and prepared for inclusion in the ERIC 
system by a network of 16 specialized clearinghouses. These 
clearinghouses acquire and process educational literature, serve user 
needs, and create information products pertaining to defined scopes 
of interest. 

These clearinghouses are responsible for: 

identifying, acquiring, and 
selecting cign ; *icant and timely 
literature within their fields; 

processing selected items for entry into the 
ERIC system by cataloguing, indexing, and 
abstracting documents; 

staying aware of current trends in 
their fields; 

preparing summary materials and state- 
of-the-art papers for policymakers, 
program planners, researchers, and 
practitioners; 



furnishing users with services and pro- 
ducts; and 

establishing and using effective 
dissemination strategies. 

Details of the issues involved in meeting these responsibilities are provided later 
in this paper. 

ERIC Processing and Reference Facility 

The ERIC Facility is a central editing and computer processing facility that coor- 
dinates acquisitions, editorial activities, document control, lexicography, data 
entry, computer processing, and reference services for the entire ERIC system. 

While the clearinghouses acquire documents within their scope of interest, the 
Facility receives all federal acquisitions and sends them to the appropriate 
clearinghouse for processing. In addition, the facility coordinates clearinghouse 
acquisitions in order to reduce duplication of efforts. 

To help ensure uniformity across clearinghouses, the Facility subjects all incoming 
records lo a standard editorial review, correcting typographical and spelling errors, 
ensuring the presence of all mandatory data fields, validating index terms against 
the authority lists, checking correctness of cataloguing, and identifying and 
resolving duplicate submissions from different clearinghouses. 

The Facility provides control for all ERIC documents. This includes providing 
standard data input forms, thesaurus input forms, reproduction releases, and 
disclaimer labels, printing and stocking a variety of systemwide brochures and 
directories, and assigning document numbers. 

The Facility coordinates the development and refinement of the ERIC Thesaurus 
and the Identnier Authority List. 

The ERIC computer system is used for compiling and refining resumes, thesaurus 
files, and Source Authority List files, and for preparing photocompositions. All 
these functions are performed by the Facility. 
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Finally, the Facility is heavily involved in responding to inquiries concerning the 
ERIC system, typically processing from 8,000 to 12,000 letters per year (Hoover 
and Brandhorst, 1982). 

ERIC Document Reproduction Service (FDRS) 

While the clearinghouses provide abstracts to various documents, EDRS provides 
users with access to the documents themselves. The EDRS contract is awarded 
by CERIC to the contractor who provides the lowest cost to the public. 

The EDRS contractor microfiches the documents after they have been processed 
by the clearinghouses and the ERIC Facility. There are now about 700 ongoing 
subscriptions to the ERIC microfiche collection. During a typical year, EDRS will 
distribute over 10 million microfiche. 

The EDRS contractor is also required to make hard copy reproductions available. 
Annual on-demand orders amount to about 50,000 reproductions. 

Database Vendors 

There are two forms of ERIC database vendors: CUE, which is published by the 
private sector, and commercial information services which provide computerized 
search and retrieval functions. 

While RIE is published by the Government Printing Office, CIJE is published by 
a contractor at no cost to the government. The publisher incurs the costs of 
printing, publishing, and marketing CIJE and recovers these costs on 
subscriptions. 

Information services obtain the ERIC database, add value in terms of search and 
retrieval systems, and market the system. Currently ERIC is available on-line 
from BRS and Dialog, and in a CD-ROM version from Silverplatter, OCLC, and 
Dialog. 



ERIC/TM Clearinghouse 



As a clearinghouse, ERIC/TM performs all the functions listed und^r "Clearing- 
houses" above. ERIC/TM's scope of interest covers: 

Tests or other measurement devices and documents whose 
major purpose is confined to discussions of such 
instruments. Such documents might announce the avail- 
ability of an instnnent, describe its characteristics, outline its 
development, colore its reliability and validity, provide a 
critical review, summarize recent developments in the use of 
the measure, or explain how it could be used more effec- 
tively. 

Methodology of measurement and evalus^on. These 
documents might describe measurement techniques or 
evaluation models, indicate the appropriate use of these 
techniques, explore their advantages and disadvantages, or 
contrast and compare various ways of collecting, quantifying, 
and analyzing data. 

Evaluation of programs, projects, and procedures. When an 
evaluation report is primarily concerned with the evaluation 
design, procedures, or instrumentation, it is within the scope 
of ERIC/TM, regardless of the population or subject area 
involved. Evaluation reports in which the methodology and 
instrumentation are secondary to programmatic concerns are 
not within ERIC/TM's scope. These documents are 
processed by the clearinghouse responsible for the subject 
area or population involved. 

Research design and methodology. These documents might 
discuss the selection of an appropriate research design, 
problems associated with various experimental or quasi- 
experimental designs, and appropriate research methods. 
Also appropriate for ERIC/TM are docun was on statistical 
procedures used to analyze research data. 
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Human development ERIQTM is responsible for thosw 
areas of human development that do not come under the 
scope of other clearinghouses. ERIQTM does not handle 
documents concerned witu the development of infants and 
young children, the handicapped, or language skills, 
mathematical reasoning, or moral values. 

Learning theory. ERIC/TI responsible for those aspects 
of learning theory that are not specific to a population (for 
example, infants and young children, the handicapped, or 
disadvantaged) or subject areas that are covered by other 
clearinghouses (for example, language acquisition or learning 
social studies concepts). 



Needed Improvements 



The evolution of information technology and the ever-changing needs of the 
educational community present real challenges to the ERIC system. The system must be 
continually evaluated and improved in order to stay current and relevant. Issues that 
now confront ERIC are discussed in two iecent reports: Final Summary Report of the 
ERIC Redesign Meeting (Johnson v 1986 v -*nd ERIC in its Third Decade (Bencivenga, 
1986). Several issues are identified in u*jse reports, including needs for: 

serving a wider audience within the educational community, 

expanding dissemination activitie for this audience, 

collaborating more clc: Jy with existing systems and 
networks, and 

controlling quality. 

Each of these concerns is described. 
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Serving a Wider Audience 



One concern with the operation of ERIC is an overly narrow focus on educational 
researchers as the target audience. Greenwood and Weiler (1972) have identified an 
expanded audience for ERIC information, including decisionmakers, teachers, 
practitioner specialists, and other disseminators, as well as basic and applied researchers. 
Nowhere is the need for an expanded audience more critical than in the area of tests, 
measurement, and evaluation. Educational improvements can only be achieved if 
methodological tools are taken out of the laboratory and applied directly to educational 
practice and policy. 

Audiences for ERIC/TM products include: 

Researchers 

The first audience of concern to ERIC/TM is educational 
researchers who specialize in developing and applying measurement 
and evaluation methodology. The information needs of this group 
have been relatively well served by ERIC7TM and by a number of 
journals dedicated to test and measurement research (for example, 
Journal ofEduc fional Measurement, Applied Psychological 
Measurement, Journal of Education Statistics, Psychometrika). 



State and Federal Policymakers 

A second audience for ERIC/TM products consists of federal 
and state officials who are involved in setting educational 
policy. This audience differs from local decisionmakers who 
have somewhat more immediate concerns and impact on 
educational pactice. State Education Agency and federal 
Department of Education officials are key members of this 
audience. 



Local Decisionmakers 

Officials who set policy for individual schools and districts 
have information needs that are somewhat distinct from those 
of state and federal policymakers. Local decisionmakers 
include superintendents, school board members, principals, 
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and in higher education, college or university presidents. The 
school board associations and some superintendents' groups 
are examples of organizations representing this constituency. 

Practitioners 

Teachers who implement particular auricular or testing 
programs have still different information needs. Teachers 
need to know more about how to interpret test scores and 
how to communicate test results to students and parents. 
The National Education Association and the American 
Federation of Teachers are examples of organizations 
representing this audience. 

Curriculum and Test Developers 

Another target audience for ERIC/TM is organizations and 
individuals who develop test materials. The National 
Association of Test Developers (NATO) and the Association 
of State Assessment Programs are examples of this audience. 

Consumers 

Consumers of educational programs, that is, taxpayers and 
parents as well as students themselves, constitute a final 
audience for ERIC/TM. This group is generally less 
organized in comparison to other ERIC/TM audiences. 
Groups such as FAIR TEST represent one segment of 
measurement consumers. Parents represent another. 
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Expanding Dissemination Activities 



A second concern in operating ERIC is the need to expand dissemination. The 
need for aggressive dissemination activities is not new. In 1963, the Presidents Science 
Advisory Committee explained a distinct ; on between a simple repository and the view of 
a clearinghouse that underlies AIR's approach. The committee said: 



The . . . depository is primarily a clearinghouse for 
documents; in general, it does not try to glean information 
from the documents it handles, but merely provides 
appropriate documents to its users. But retrieval of 
documents is not the same as retrieval of information. . . To 
retrieve information, as opposed to documents, the technical 
community has devised the specialized data and information 
center. 

A specialized information center makes it its business to 
know everything that is being published in a given field ... it 
collates and reviews the Jata, and provides its subscribers 
with regularly issued compilations of critical reviews, 
specialized bibliographies, and other such tools. Its input is 
the output of the central depository. 



ERIC clearinghouses should play the role of information broker. In addition to 
acquiring documents, clearinghouses should play an active role in transforming that 
information to a form others can use. 



Collaborating More Clos ely with Existing Systems 

Another operational issue that the clearinghouse contractors must address is how 
best to increase the effectiveness of the ERIC system within the limits of constrained 
federal resources. The clearinghouses must coordinate closely with existing groups and 
systems in order: 



to extend their effectiveness in gathering relevant documents, 



to extend their effectiveness in disseminating information 
analysis products, 
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to avoid costly duplication of efforts. 



ERIC needs to multiply federal dollars through collaboration with existing groups 
and systems. 

Quality Control 

A final operational issue for ERIC/TM and the ERIC system as a whole is 
quality control. The willingness of potential users to invest their time and resources to 
obtain ERIC information is a direct function of the quality of the information that they 
come to expect. Too often individuals conduct a search only to find important 
documents are not in the database. ERIC/TM must be vigilant in selecting documents 
to weed out irrelevant or technically flawed material. Each selected document must be 
accurately abstracted and indexed so that users can quickly determine if it is relevant to 
their needs. Information analysis products must be successful in meeting their intended 
objectives. 
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Changes in ERIC/TM 



As a result of a contract competition, the ERIC Clearinghouse on Tests, 
Measurement, and Evaluation was transferred from the Educational Testing Service to 
the American Institutes for Research in January 1988. ETS was the first contractor and 
it managed the clearinghouse for 18 years. 

With new management and staff, AIR is building on the work of ETS and has 
established the following goals for ERIC/TM: 

1) improved document acquisition and processing. ERIC/TM is meeting this goal by: 

acquiring relevant documents more aggressively, 
processing the acquired documents faster, 
developing more informative and helpful abstracts, 
and 

having experts in measurement and evaluation oversee 
operations more closely. 

2) better quality and more relevant products. ERIC/TM is meeting this goal by: 

establishing an active advisory board with broader 
representation, 

developing targeted products for more audiences, 
. fjhasizing the needs of practitioners and policy 
'i; cers, and 

incorporating more peer review. 

3) improved dissemination of information. ERIC/TM is meeting this goal by: 

utilizing existing networks better, and 
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establishing a network of ERIC affiliates. 



Dement Atq^iaLte 

Acquisition is the critical first step in building the database. ERIQTM uses six 
approaches to gathering documents for consideration for inclusion in the ERIC system: 

1. Standing Acquisition Arrangements - ongoing agreements with 
various organizations and conference coordinators in which they 
automatically send documents for ERIC review. ERIQTM is 
constantly seeking to add more organizations to the list of 
organizations with standing acquisition arrangements. 

2. Monitoring programs of meetings - ERIQTM examines these 
programs to identify papers potentially within the scope of interest 
and contact appropriate parties. 

3. Monitoring newsletters and journals - Newsletters from state 
departments of education, research centers, professional organiza- 
tions, and proprietary organizations frequently contain references to 
new reports and releases appropriate for processing by ER1C/TM. 

4. Bibliographies in documents and journals - ERIC/TM examines 
bibliographies, references, and notes in reports to identify items of 
potential interest to RIE. 

5. Unsolicited documents - ERIC7TM encourages individuals to 
submit documents for ERIC/TM review by stating desires in 
correspondence, brochures, and announcements. 

6. Journal subscriptions - ERIC/TM subscribes to 28 journals in the 
areas of testing, evaluation, research, and learning theory. Due to 
funding limitations, coverage of these journals is incomplete. Most 
of the articles in seven journals are abstracted for CIJE. Selection 
of articles for the other journals is based on the articles contribution 
of substantive new material to the body of education information, it 
relationship to the field of education, and its comprehensiveness. 
Book reviews, editorials, letters to the editor and the like are not 
normally selected. In 1988, ERIC/TM added three journals to its 
processing list: Applied Psychological Measurement (APM), 
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Evaluation Practice, and Applied Measurement in Education. Past 
thematic issues of APM were also processed. 



Details of document acquisition can be found in Hannaman (1989). 

Selection of documents 

All documents ERIQTM acquire for consideration for inclusion in RIE are eval- 
uated using high standards of quality. These standards, which are described in 
Eissenberg (1989), include: 

technical adequacy, 

contribution to knowledge and significance, 
relevance, 

new applications of knowledge and innovative practices, 
effectiveness of presentation and thoroughness of reporting, 
responsiveness to current priorities, 
timeliness, 

authority of author, source, or sponsor, and 
comprehensiveness. 

Journal articles are evaluated for their relationship to the field of education and 
comprehensiveness as an article (due to funding limitations, book reviews, editorials, 
letters to the editor and the like are not normally selected.) 

Acquisitions are reviewed by the Review Coordinator and the Director. As a 
check for inter-reviewer consistency, meetings are held periodically to review selection 
decisions. Additionally, ERIQTM sends 10% to 20% of the documents out for external 
review and compares these results with its internal evaluations. The results of two such 
evaluations can be found in Eissenberg and Rudner (1988) and Eissenberg (1989). 

Quality of abstracts and indexing 

While virtually everyone who has ever written a paper has prepared an abstract, 
there is a science and art to abstracting that is well recognized among information 
scientists. 

The field of abstracting makes a distinction between informative abstracts and 
indicative abstracts. An informative abstract summarizes the information that is in the 
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report. An indicative abstract states what the report is about, but it doesn't provide 
conclusions. Consider, for example, the following abstract; 

"A biochemistry examination administered to 114 dental 
students consisted of 20 items that measured knowledge and 
18 items that measured either comprehension or application. 
Each item was written with five choices, half the class being 
presented the original version, half a content-similar that 
included three choices.* 9 

This is neither fish nor fowl. This starts as an informative abstract providing 
specific details of the study and ends almost as an indicative abstract implying what is in 
the paper. It is not helpful since it does not tell us whether student performance varied 
as a result of item format. 

The majority of ERIC Gearinghouses hire staff to serve as abstractors and 
indexers. The theory is that these individuals are experts within the scope of the 
clearinghouse and that they can learn abstracting. The practice is quite different. 
Abstractors and Indexers tend to be master's level graduate students with little expertise 
in the clearinghouse field. Once they master the art and science of abstracting they tend 
to leave the clearinghouse for better incomes as professional abstractors. As a result, 
clearinghouses have a problem with staff turnover and tend to spend a great deal of time 
training abstractors (Erickson, 1986). Training abstractors is an area clearinghouses are 
not equipped to do and do not do well. 

ERIQTM is unique among ERIC Clearinghouses in that it now uses an outside 
professional abstracting firm, Herner and Company, one of the oldest library and 
information science consulting firms in the country, supervises abstractors and implement 
rigorous quality control procedures. Herner and Company has abstracted over 2,000,000 
items since its founding. They are quite familiar with the ERIC system as they 
conducted some of the research upon which CIJE is based. As abstractors for the 
American Psychological Association, they are familiar with abstracting documents in 
measurement and technical fields. 

The following measures are intended to ensure the highest possible quality of 
abstracting for ERIQTM: 

Use only abstractors with at least 5 years of experience in: 

preparing informative abstracts, 

editing abstracts to conform with the American National 
Standards Institution (ANSI) standards, and 
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indexing based on Thesauri such as the ERIC Thesaurus of 
Descriptors. 

Sequentially evaluate all abstracts and indexes to detect and remedy 
errors and to identify and remedy recurring patterns of errors arising 
from shortcomings or misunderstandings. 

Perform computer spelling verification of all abstracts and index 
records. 

Examine all records through a computer-based diagnostic program 
that checks for correct field tags, correct field lengths, correct data 
types (e.g., numeric data in a year field), and total number of words. 

Have a human proofread and correct spelling-verified and field- 
diagnosed records to correct undetected syntactic and field entry 
errors. 



Advisory Board 

The National Advisory Board provides professional subject guidance and counsel 
on the direction of the clearinghouse programs, scope, document selection criteria, 
information analysis topics, and matters concerning the dissemination of educational 
information. The advisory board 

evaluates potential information analysis products (IAPs) and set 
priorities among alternatives, 

evaluates the quality of the IAPs, 

reviews progress in processing documents for RIE and CUE and 
evaluate effectiveness in identifying, selecting, and abstracting 
relevant materials, and 

discusses long-range objectives for the operation of the clearin- 
ghouse. 

ERIC/TM has established a proactive advisory board representing a variety of 
organizations and a variety of information needs. Specifically, the board representative 
of basic researchers, state level policy makers, local level policy makers, teachers, 
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program planners, consumer advocates, education writers, and test publishers. While 
ERIC/TM has extensive contact with the research community and teachers, there is a 
need for representation from these groups on the Advisory Board. 

The Advisory Board members are brought to ERIQTM for an annual meeting. 
Prior to the meeting, subgroups of two advisors a~e asked to review LAPs, abstracts, and 
other products. These individuals lead subcommittee meetings during lunch. 



Product Lin? 



ERIC/TM is trying to establish a clearinghouse that is truly an information center 
and not just a simple repository of documents. Toward that end, ERIC/TM has 
established a product line featuring the following: 



References on Disk - Cumulative indices of Journal of Educational 
Measurement, Educational Measurement, Journal of Educational 
Statistics, Psychometrika, American Educational Research Journal, 
Review of Educational Research on diskette along with descriptors 
and search software. 

Monographs - Clearinghouse sponsored publications such as 
research reviews, papers summarizing a given topic, state of the art 
papers, and guides to the measurement literature. This year 
ERIC/TM is working with the Buros Institute to prepare a guide to 
understanding district level test scores and with the Lawyers 
Committee for Civil Rights Under Law to prepare a popular piece 
on legal rights with regard to test and evaluation information. 



Digests - short, two page, documents which synthesize various test, 
measurement and evaluation topics and are targeted to various 
audiences. Through direct mail, online systems, as well as contacts 
with publisher's representatives, the National Diffusion Network, 
journal and newsletter publishers, and the Education Writer's 
Association, Digests are widely distributed. 



Highlights - These are bibliographies and abstracts on hot topics in 
measurement. 
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Quality of Information Analysis Products 



An extensive needs assessment is conducted to identify potential topics for 
clearinghouse information analysis products. The following input is considered: 

advisory board member recommendations, 
user requests for information, 

analysis of RIE and CIJE to identify trends and issues, 
solicitation for topics from the field, 

assessment of topics at professional meetings and in newsletters and 
periodicals, 

priorities of the Department of Education, and 
formal and informal contact 0 with the field. 

Each year ERIQTM develops a proposed list of products for the advisory board 
to consider. ERIQTM then sends the list of products to Central ERIC for approval. 

In developing every product, ERIOTM follows tne process model of document 
design. Authors for the products are selected based on a review of recommendations 
from the advisory board, review of the literature, consultation with other organizations, 
external recommendations, and AIR's own experiences. 

Once products are developed, they undergo an extensive peer and audience 
review. Input from teachers, school administrators, and other school-based personnel as 
well as the measurement research community. 

On an annual basis, the products are reviewed by the advisory board. As with the 
other reviewers, the advisory board will evaluate products for timeliness, significance, 
accuracy, and appropriateness for the intended audience. 

ERIC/TM Affiliates 

ERIQTM is establishing a broad network of individual affiliates. They help us 
review acquisitions, prepare documents, evaluative clearinghouse products, and locate 
documents and other data sources. ERIC/TM is looking for additional vo^nteers to 
serve as affiliates. 
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Collaborative Activities 



As many reviewers of ERIC have noted, to be most useful, clearinghouses must 
capitalize on existing resources (Plank, 1986; Bencivenga, 1987). ERIC/TM is working 
collaboratively with different organizations and individuals to aid »n 



acquiring documents 
preparing materials 
disseminating information, ai I 
providing feedback. 
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Concluding remarks 



The ERIC Clearinghouse on Tests, Measurement, and Evaluation (ERIC/TM) 
has acquired, reviewed, and processed more than 20,000 journal articles, conference 
papers, reports, and other papers over the past 22 years. On an annual basis, ERIC/TM 
is now producing 14-16 literature syntheses per year. As pari of the larger ERIC system, 
ERIC/TM has become a major resource for policy makers, program planners, 
researchers, teachers, and others. Over 3.000 organization provide access to ERIC/TM 
products. 

ERIC does not optimally serve the measurement community. Individuals 
frequently conduct ERIC searches only to find that key papers that siiould be in the 
system are not there. While ERIC/TM has access to much of the literature pertaining 
to tests, measurement, and evaluation, very little has been synthesized or transformed 
into tools serving different audiences. 

In order to better serve policy makers, program planners, researchers, teachers, 
and others, ERIC/TM needs to involve more organizations and more individuals. 
Higher quality and more relevant documents need to be obtained from authors and 
added to the ERIC database. Information that is common knowledge for some 
audiences needs to be transformed into information that is useful for other audiences. 
Existing dissemination channels that are effectively communicating with their audiences 
need to be provided with high quality targeted information. While ERIC/TM is 
establishing a structure to enable these activities, the success of ERIC/TM Ultimately 
relies on the cooperation and involvement of the measurement community. 
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